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Art Unit: 2624 

Response to Arguments 

Applicant's amendments/remarks filed 12/12/2008 have been entered and made 
of record. With respect to claims 1 and 6, the applicant argues on remarks page 
13 and 15 that Hsu does not disclose the feature points, the examiner disagrees. 
Hsu at Col 4 lines 65 "map points between images" which clearly shows that 
feature points are mapped between the images. Hsu further discloses at Col 6 
lines 47-55 where the correspondence between the points in the images which 
are the combination of pixels (brightness/color pixels which are considered 
as feature points of the image). Hsu further discloses the color 
matching/blending module 31 0 at lines 52-54 where the color (feature of the 
images) are mapped between the images. Hsu further discloses overlap 
between the images (typically four points) at (Col 13 lines 18-20) which can be 
interpreted as feature points. Thus from all of the above explanation Hsu clearly 
discloses the feature points and the correspondence between them in the 
images. The claim does not limit as to what type of feature points etc. 
With regards to claim 14-15 the applicant is arguing the computer 
readable medium which as in the specification page 5, line 15 (the third 
full paragraph on page 5) "data signals that include such computer 
programs and are embodied in carrier waves" which encompasses the 
non statutory subject matter. The applicant is required to claim only the 
tangible computer readable media such as one disclosed on page 22 lines 
19-25 which represents tangible computer readable recording or storage 
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medium. Thus the claims 14-15 should be amended with "a tangible 
computer readable storage medium". 

Regarding claim limitations "— executed by an integrated circuit" in claims 
1,6,7,12-13, the examiner suggests amending the claims" — executed by a 
processor" because there are "integrated circuits" such as "analog integrated 
circuits" such as operational amplifiers, sensors etc. which do not have 
computational processing capacities. 

Specification 

The amendment filed on 12/12/2008 is objected to under 35 U.S.C. 132(a) 
because it introduces new matter into the disclosure. 35 U.S.C. 132(a) states 
that no amendment shall introduce new matter into the disclosure of the 
invention. The deletion of "and data signals that include such computer 
programs and are embodied in carrier waves" changes the scope of the invention 
and introduces new matter situation. The applicant is advised to undo the 
amendments to the specification. The specification should therefore read as 
follows and as originally filed. 

Paragraph beginning at page 5, line 15 (the third full paragraph on page 
5), with the following rewritten paragraph as originally filed on 04/13/2005: 



The technique of the present invention is actualized by a variety of 
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applications, which include image processing methods, computer 
programs that attain the functions of such apparatuses and methods, and 
recording media in which such computer programs are recorded, and data 
signals that include such computer programs and are embodied in carrier 
waves. 

Claim Rejections - 35 USC §112 

The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

Claims 1-5 are rejected under 35 U.S.C. 112, first paragraph, as failing to comply 
with the written description requirement. The claim(s) contains subject matter 
which was not described in the specification in such a way as to reasonably 
convey to one skilled in the relevant art that the inventor(s), at the time the 
application was filed, had possession of the claimed invention. The newly added 
limitation "determined correspondence relationship in a three dimensional space" 
is not fully disclosed, described and supported by the specification. The 
specification on page 2 lines 15-16 does not describe the above claimed 
limitation. Figs 6 and 8 disclose the coordinate conversion; however they are for 
cylindrical coordinate system and do not support the spherical coordinate system 
or the three dimensional space claimed in claim 1 . Claims 2-5 depend directly or 
indirectly on claim 1 and therefore they are rejected. 



Application/Control Number: 10/531,192 
Art Unit: 2624 



Page 5 



Claim Rejections - 35 USC § 101 

35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

The USPTO "Interim Guidelines for Examination of Patent Applications for Patent 
Subject Matter Eligibility" (Official Ga zette notice of 22 November 2005 ), Annex 
IV, reads as follows: 

Descriptive material can be characterized as either "functional descriptive material" or "nonfunctional 
descriptive material." In this context, "functional descriptive material" consists of data structures and 
computer programs which impart functionality when employed as a computer component. (The 
definition of "data structure" is "a physical or logical relationship among data elements, designed to 
support specific data manipulation functions." The New IEEE Standard Dictionary of Electrical and 
Electronics Terms 308 (5th ed. 1993).) "Nonfunctional descriptive material" includes but is not limited 
to music, literary works and a compilation or mere arrangement of data. 

When functional descriptive material is recorded on some computer-readable medium it becomes 
structurally and functionally interrelated to the medium and will be statutory in most cases since use of 
technology permits the function of the descriptive material to be realized. Compare In re Lowry, 32 
F.3d 1579, 1583-84, 32 USPQ2d 1031, 1035 (Fed. Cir. 1994) (claim to data structure stored on a 
computer readable medium that increases computer efficiency held statutory) and Warmerdam, 33 
F.3d at 1 360-61 , 31 USPQ2d at 1 759 (claim to computer having a specific data structure stored in 
memory held statutory product-by-process claim) with Warmerdam, 33 F.3d at 1361, 31 USPQ2d at 
1760 (claim to a data structure per se held nonstatutory). 

In contrast, a claimed computer-readable medium encoded with a computer program is a computer 
element which defines structural and functional interrelationships between the computer program and 
the rest of the computer which permit the computer program's functionality to be realized, and is thus 
statutory. See Lowry, 32 F.3d at 1583-84, 32 USPQ2d at 1035. 



Regarding claims 14-15 recite a computer program product which are essentially 
a programs (software program) and programs per se are non statutory. The 
Examiner suggests claiming the tangible computer readable storage medium and 
not the program. Please amend the claims 14-15 to include "A tangible computer 
readable storage medium" at lines 6 in the body of the claim. 



Claim Rejections - 35 USC § 103 
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The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1-3,5 ,12 and 14 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hsu et al (US 6078701 ) hereafter Hsu in view of Chen et al 
(US 6486908) hereafter Chen as best understood by the examiner due to 1 12 
rejections. 

1. Regarding claim 1, Hsu discloses an image processing apparatus (Figs 1-3 , 
an image processor which has integrated circuits executing the steps) for 
generating graphics data representing a single seamless planar image 
synthesized from a multiple sets of graphics data contained in a plurality of 
graphics files, in response to the plurality of graphics files each of which contains 
the graphics data composed of a multiplicity of planar pixels arrayed in a plane 
for representing a planar image, the image processing apparatus comprising: 

a feature point extractor configured to extract a feature point which is an 
area having a predetermined characteristic, from each of the plurality of 
spheroidal images (one to one correspondence with points in the scene at 
Col 1 lines 48-49,Col 4 Lines 65 mapping points between image frames are 
the features extracted from the images and Col 6 line 54-55 where each 
point is in one-to-one correspondence); 



Application/Control Number: 1 0/531 ,1 92 Page 7 

Art Unit: 2624 

a correspondence relationship determiner configured to determine a 
correspondence relationship of the extracted feature points, between the plurality 
of spheroidal images (Fig 9 and Col 6 lines 36-67 where the mapping is 
explained); 

a spheroidal image synthesizer configured to generate seamless 
spheroidal graphics data representing a single seamless spheroidal image, by 
synthesizing a plurality of graphics data each of which representing each of the 
spheroidal images, with reference to the determined correspondence relationship 
in a three dimensional space (Fig 9, Col 14 lines 1-10 and Col 17 lines 43-62 
where the three dimensional spherical mosaics are generated from two 
dimensional image data showing the relationship in three dimensional 
space); and 

a planar image generator configured to generate the graphics data 
representing the single seamless planar image, from the seamless spheroidal 
image graphics data (Col 12 lines 63 through col 13 lines 57 where reference 
to image mappings constitutes the planar image from the spherical 
reference). Hsu discloses area of the mosaic based on the perspective 
projection at Col 6 lines 36-67 and Col 14 lines 1-5, however does not explicitly 
recite a synthesis area establisher configured to establish a spheroidal projection 
plane centered on a predetermined point, as an area for synthesis of the multiple 
sets of graphics data and a spheroidal image generator configured to generate a 



Application/Control Number: 1 0/531 ,1 92 Page 8 

Art Unit: 2624 

plurality of spheroidal images, by projecting each of planar images represented 
by each of the multiple sets of graphics data onto the projection plane. 

Chen discloses a synthesis area establisher configured to establish a 
spheroidal projection plane centered on a predetermined point, as an area for 
synthesis of the multiple sets of graphics data and a spheroidal image generator 
configured to generate a plurality of spheroidal images, by projecting each of 
planar images represented by each of the multiple sets of graphics data onto the 
projection plane (Figs 13,14 and 19 where the area or the rectangle 
determines the number of images can fit and a rectangular (planar) image 
can be used to represent the spherical environment map at Col 6 lines 42- 
43, abstract also discloses the number of photographs required using the 
center point of each photographs). Chen discloses an apparatus (Fig 1 
apparatus with processor which has integrated circuits for processing) and 
further discloses that the method and system as disclosed builds the spherical 
panoramas based on the focal length of the camera, the height of the film and 
the overlapping ratio between the films thus avoiding the use offish eye lens 
giving a better panorama at Col 2 lines 39-50. Hsu and Chen together would 
meet the limitations of wherin each of the synthesis area establisher, the 
spheroidal image generator, the feature point extractor, the correspondence 
relationship determiner, the spheroidal image synthesizer, and the planar image 
generator is executed by an integrated circuit (processors). Hsu and Chen are 
from the same field of endeavor and are analogous art, therefore it would be 
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obvious for one of ordinary skill in the art at the time the invention was made to 
use the teachings of Chen in the apparatus of Hsu for the above reasons. 



2. Regarding claim 2, Hsu and Chen disclose the image processing apparatus in 
accordance with claim 1 . Chen further disclose wherein the plurality of graphics 
files further include image attribute information which is attribute information of 
the graphics data (film height a, film width b, and focal length are the 
attributes Col 7 lines 20-25), a focal length distance determiner being executed 
by an integrated circuit (Fig 1 processor), wherein the image processing 
apparatus further comprises a focal distance determiner configured to determine 
a focal distance of an optical system used to generate the multiple sets of 
graphics data for each of the multiple set of graphics data (focal length of the 
camera system col 2 lines 64), in response to the image attribute information; 
and the spheroidal image generator generates the plurality of spheroidal images 
by projecting each planar image represented by each of the multiple sets of 
graphics data onto the projection plane, the each planar images being placed at 
a location away from the predetermined point to the projection plane side, by the 
focal distance corresponding to each of the multiple sets of graphics data (the 
number of circles landscape is the planar determination of the images 
using the focal length of the camera at col 2 lines 53-60). 



3. Regarding claim 3, Hsu and Chen disclose the image processing apparatus in 
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accordance with claim 2. Chen discloses further the focal distance of the camera 
at (Col 2 lines 53 through Col 3), focal plane resolution, pixel size width and 
heights at (Col 6 lines 41-56). 

4 Regarding claim 5, Hsu and Chen discloses and apparatus in accordance to 
claim 2. Chen discloses further the focal plane resolutions at (Col 6 Lines 12- 
56). Chen discloses further the focal length and the angle (180 degree) divided 
by the focal length (f) at (Col 7 Lines 47-50). 

5. Claim 12 is a corresponding method claim of claim 1. See the explanation of 
claim 1. 

6. Claim 14 is a corresponding computer program product claim of claim 1 . See 
the explanation of claim 1 . 

Claims 6, 8 and 13 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Lipscomb (US 6031541) hereafter Lipscomb in view of Hsu. 

7. Regarding Claim 6, Lipscomb discloses an image processing apparatus (Fig 
1) for generating graphics data representing a single seamless planar image 
synthesized from a multiple sets of graphics data contained in a plurality of 
graphics files, in response to the plurality of graphics files each of which contains 
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the graphics data composed of a multiplicity of planar pixels arrayed in a plane 
for representing a planar image, the image processing apparatus comprising: 

a synthesis area establisher configured to establish a cylindrical projection 
plane centered on a predetermined axis, as an area for synthesis of the multiple 
sets of graphics data (Fig 12 which shows the number of pictures needed 
and constitutes the area of the mosaic, Figs 3 and 4 shows multiple 
rectangular areas and the axis and projections with respect to the cylinder); 

a cylindrical image generator configured to generate a plurality of 
cylindrical images, by projecting each of planar images represented by each of 
the multiple sets of graphics data onto the projection plane (Figs 5 and 12 which 
shows multiple cylindrical images projected and aligned in form of a 
mosaic in fig 12); a cylindrical image synthesizer configured to generate 
seamless cylindrical graphics data representing a single seamless cylindrical 
image, by synthesizing a plurality of graphics data each of which representing 
each of the cylindrical images, with reference to the determined correspondence 
relationship in a three dimensional space (Fig 13 shows the mapping of the 
input image into the cylindrical coordinates which is a three dimensional 
spacem, Col 6 lines 55-60); and a planar image generator configured to 
generate the graphics data representing the single seamless planar image, from 
the seamless cylindrical image graphics data at (Col 6 lines 21-30 where the 
cylinder is "unrolled" to for a large planar rectangle consisting of the 
panoramic graphics image). Lipscomb discloses the color extraction (color of 
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the panoramic scene at Col 2 line 67 through Col 3 lines land Col 10 lines 
35-47 explains the color features) and the correspondence mapping at (Fig 20 
elements 205-207 the correspondence between the images are determined), 

however does not expressly recite a feature point extractor configured to extract 
a feature point which is an area having a predetermined characteristic, from each 
of the plurality of cylindrical images and a correspondence relationship 
determiner configured to determine a correspondence relationship of the 
extracted feature points, between the plurality of cylindrical images. 

Hsu discloses a feature point extractor configured to extract a feature 
point which is an area having a predetermined characteristic, from each of the 
plurality of cylindrical images and a correspondence relationship determiner 
configured to determine a correspondence relationship of the extracted feature 
points, between the plurality of cylindrical images (one to one correspondence 
with points in the scene at Col 1 lines 48-49,Col 4 Lines 65 mapping points 
between image frames are the features extracted from the images, Col 4 
lines 56-57 where cylindrical surface is disclosed and Col 6 line 54-55 
where each point is in one-to-one correspondence). Lipscomb discloses an 
apparatus (Fig 1 apparatus with (cpu) processor which has integrated 
circuits for processing) and further discloses that the method and system as 
disclosed builds the cylindrical panoramas as seen in Fig 12. Hsu discloses the 
apparatus with integrated circuits at (Figs 1-3 , an image processor which has 
integrated circuits executing the steps). Lipscomb and Hsu together would 
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therefore meet the limitations of wherin each of the synthesis area establishes 
the cylindrical image generator, the feature point extractor, the correspondence 
relationship determiner, the cylindrical image synthesizer, and the planar image 
generator is executed by an integrated circuit (processors). Hsu discloses that 
the system and method as disclosed accurately constructs seamless panoramic 
image from the sequence of images using topology, global registration at (Col 2 
Lines 53-60). Therefore it would be obvious for one of ordinary skill in the art at 
the time the invention was made to have used the teachings of Hsu in the 
apparatus of Lipscomb to achieve the claimed invention. 

8. Regarding claim 8, Lipscomb and Hsu disclose the image processing 
apparatus in accordance with claim 6. Lipscomb discloses further wherein the 
cylindrical image generator establishes the axis parallel to the height direction 
established in the graphics data (Fig 18 where the axis of the cylinder is 
parallel to the height of the images). 

9. Claim 13 is a corresponding method claim of claim 6. See the explanation of 
claim 6. 

Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over Hsu in 
view of Chen and in further view of Muramatsu. (US 5438380) hereafter 
Muramatsu. 
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10. Regarding claim 4, Hsu and Chen disclose the image processing apparatus 
in accordance with claim 2. Hsu and Chen both disclose photographic films (for 
eg Chen Col 3 Lines 59-67). Chen further discloses the spheroidal image 
generator determines 35 mm film size as a size of the planar image (Figs 13,14 
and 19 where the area or the rectangle determines the number of images 
can fit and a rectangular (planar) image can be used to represent the 
spherical environment map at Col 6 lines 42-43, abstract also discloses the 
number of photographs required using the center point of each 
photographs), discloses the focal length of the camera system and the film 
height and the width (at col 3 Lines 1-18) however do not expressly recite 
wherein the image attribute information includes 35 mm-equivalent lens focal 
distance which is a value of focal distance converted to a 35 mm film camera 
basis; the focal distance determiner determines the 35 mm-equivalent lens focal 
distance to be the focal distance. 

Muramatsu discloses the focal distance (length) of the film from 25mm to 
35 mm and the image reproduced (Col 2 lines 18). Muramatsu discloses that the 
camera provides aspect ratio of good appearance of the photographic images 
when used in panoramic format (Col 1 lines 40-45 and 55-56). Muramatsu, Hsu 
and Chen are from the same field of endeavor and are analogous art, therefore it 
would be obvious for one of ordinary skill in the art at the time the invention was 
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made to have used the teachings of Muramatsu in the apparatus of Hsu and 
Chen for the above reasons. 



Claims 7, 9, 1 1 and 15 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Lipscomb in view of Hsu and in further view of Chen. 



11. Regarding claim 7, Lipscomb and Hsu disclose the image processing 
apparatus in accordance with claim 6. Lipscomb discloses the attributes of the 
images such as height and width of the images (Figs 18,19) being used in the 
panorama construction. Lipscomb and Hsu are silent and however do not 
expressly disclose wherein the plurality of graphics files further include image 
attribute information which is attribute information of the graphics data, wherein 
the image processing apparatus further comprises a focal distance determiner 
configured to determine a focal distance of an optical system used to generate 
the multiple sets of graphics data for each of the multiple set of graphics data, in 
response to the image attribute information; and the cylindrical image generator 
generates the plurality of cylindrical images by projecting each planar image 
represented by each of the multiple sets of graphics data onto the projection 
plane, the each planar images being placed at a location away from the 
predetermined axis to the projection plane side, by the focal distance 
corresponding to each of the multiple sets of graphics data. 
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Chen discloses the focal length of the camera system (Col 3 lines 10 
computed using an integrated circuit (Fig 20 processor)) and constructs a 
warped images of each of the photographic images using the attributes (color) at 
(Col 3 lines 55-60, Col 12 lines 1-32) and thus constructing a seamless 
spherical image from the warped images and thus one of ordinary skill in the art 
would be able to extend the teachings of Chen in construction of cylindrical 
panoramas. Even the height and width of the film can be understood as the 
attributes. Chen discloses that the method and system as disclosed builds the 
spherical panoramas based on the focal length of the camera, the height of the 
film and the overlapping ratio between the films thus avoiding the use offish eye 
lens giving a better panorama at Col 2 lines 39-50. Lipscomb, Hsu and Chen are 
from the same field of endeavor and are analogous art, therefore it would be 
obvious for one of ordinary skill in the art at the time the invention was made to 
use the teachings of Chen in the apparatus of Lipscomb and Hsu for the above 
reasons. 

12. Regarding claim 9, Lipscomb, Hsu and Chen disclose the image processing 
apparatus in accordance with claim 7. Lipscomb discloses further pixel or image 
height at (Col 12 lines 1-3), pixel width direction at (Col 13 lines 1-12). Chen 
also discloses the focal plane resolutions at (Col 6 Lines 12-56). 
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13. Regarding claim 1 1 , see the explanation of claim 9 and also see the angle 
over width calculation at (Col 13 line 42 in Lipscomb). 

14. Claim 15 is a corresponding computer program product claim of claim 6. See 
the explanation of claim 6. 

Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Lipscomb, Hsu, Chen and in further view of Muramatsu. 

15. Regarding claim 10, Lipscomb, Hsu and Chen disclose the image 
processing apparatus in accordance with claim 7. Lipscomb, Hsu and Chen 
discloses the photographic film however do not expressly recite 35 mm film size. 
Muramatsu discloses the focal distance (length) of the film from 25mm to 35 mm 
and the image reproduced (Col 2 lines 18). Muramatsu discloses that the 
camera provides aspect ratio of good appearance of the photographic images 
when used in panoramic format (Col 1 lines 40-45 and 55-56). Muramatsu, 
Lipscomb, Hsu and Chen are from the same field of endeavor and are analogous 
art, therefore it would be obvious for one of ordinary skill in the art at the time the 
invention was made to have used the teachings of Muramatsu in the apparatus 
of Lipscomb, Hsu and Chen for the above reasons. 
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Other Cited Prior art 

The other cited prior art made of record but not relied on are (US 
6532037), (US 6011558), (US 5396583), (US 6002430),(US 20030117488), (US 
6028584), (US 596321 3),(US 6891561), (US 20020154812), (US 6995790) and 
(US 20040247173). 

Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 
37 CFR 1 .136(a). A shortened statutory period for reply to this final action is set 
to expire THREE MONTHS from the mailing date of this action. In the event a 
first reply is filed within TWO MONTHS of the mailing date of this final action and 
the advisory action is not mailed until after the end of the THREE-MONTH 
shortened statutory period, then the shortened statutory period will expire on the 
date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .1 36(a) will be calculated from the mailing date of the advisory action. In 
no event, however, will the statutory period for reply expire later than SIX 
MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to JAYESH A. PATEL whose telephone number 
is (571)270-1227. The examiner can normally be reached on M-F 7.00am to 
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4.30 pm (5-4-9). If attempts to reach the examiner by telephone are 

unsuccessful, the examiner's supervisor, Brian Werner can be reached on 571- 

272-7401 . The fax phone number for the organization where this application or 

proceeding is assigned is 571-273-8300. Information regarding the status of an 

application may be obtained from the Patent Application Information Retrieval 

(PAIR) system. Status information for published applications may be obtained 

from either Private PAIR or Public PAIR. Status information for unpublished 

applications is available through Private PAIR only. For more information about 

the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on 

access to the Private PAIR system, contact the Electronic Business Center 

(EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO 

Customer Service Representative or access to the automated information 

system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

03/26/2009 
/Jayesh A Patel/ 
Examiner, Art Unit 2624 



/Brian P. Werner/ 

Supervisory Patent Examiner, Art Unit 2624 



